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Abstract 

Information  Technology  (IT)  solutions  to  patient  safety/medical  error  problems 
are  promising,  but  only  in  the  early  stages  of  their  development  and 
implementation.  Many  current  IT  applications  for  patient  safety  are  focused  on 
hospital  processes,  such  as  physician  order  entry  and  clinical  information  system- 
based  alerts  for  critical  values.  These  interventions  are  broadly  based,  often 
targeting  all  medications  ordered  or  all  critical  values  for  all  patients,  and,  by 
necessity,  are  not  specifically  configured  to  address  the  care  of  patients  with  a 
particular  condition  or  clinical  problem.  However,  a  condition-  or  problem- 
oriented  IT  system  might  better  integrate  into  the  care  of  frequent  conditions  and 
thereby  be  more  effective.  Accordingly,  the  authors  focused  on  the  use  of  IT  for 
the  most  common  serious  conditions  requiring  emergency  and  acute  care — the 
diagnosis,  triage,  and  treatment  of  emergency  department  (ED)  patients  with 
potential  acute  coronary  syndrome  (ACS).  This  condition  includes  acute 
myocardial  infarction  (AMI),  also  known  as  acute  cardiac  ischemia  (ACI),  and 
unstable  angina  pectoris  (UAI),  a  condition  that  can  lead  to  AMI.  The  Acute 
Cardiac  Ischemia  Time-Insensitive  Predictive  Instrument  Information  System 
(ACI-TIPI-IS)  Demonstration  Project  at  Tufts-New  England  Medical  Center  used 
multiple  IT  applications  for  patient  safety,  combining  real-time  decision  support, 
alerting,  and  retrospective  feedback  for  perfonnance  improvement,  all  for  the  care 
of  patients  presenting  to  the  ED  with  symptoms  suggestive  of  ACS.  A  Web-based 
relational  database  system,  the  TIPI-IS,  electronically  compiled  information  from 
existing  operational  systems  about  all  ED  patients  for  whom  an  electrocardiogram 
(ECG)  was  done  for  possible  ACS  symptoms,  and  supported  the  patient  safety 
aspects  of  the  project.  Real-time  decision  support  was  provided  by  the  automatic 
printing  of  the  probability  of  ACS  on  the  ECG,  as  calculated  by  the  ACI-TIPI. 
Various  types  of  physician-specific  alerts  or  retrospective  feedback  reports  about 
the  management  and  outcomes  of  patients  with  potential  ACS,  such  as  for  patients 
with  abnormal  cardiac  biomarkers  discharged  to  home,  were  produced  by  the 
TIPI-IS  and  provided  online  via  e-mail-linked  access,  and  via  paper  fonnats,  to 
support  self-evaluation  of  staff,  practice  improvement,  and  ED  operations 
oversight.  Through  the  successful  implementation  and  demonstration  of  the 
system,  the  authors  identified  barriers  and  lessons  learned  that  will  inform  the 
continued  development  of  this  IT  approach  for  preventing  patients  with  missed 
ACS  from  being  overlooked  in  the  ED. 
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Introduction 

Acute  coronary  syndrome  (ACS)  and  the  synonymous  condition  acute  cardiac 
ischemia  (ACI),  both  of  which  include  acute  myocardial  infarction  (AMI)  and 
unstable  angina  pectoris  (UAP)  that  can  lead  to  AMI,  are  the  most  common 
serious  conditions  requiring  emergency  and  acute  care.  Among  the  7  million 
patients  who  present  to  emergency  departments  (EDs)  in  this  country  each  year 
with  symptoms  consistent  with  a  cardiac  problem,  about  25  percent  will  prove  to 
have  true  ACS.  About  a  third  of  the  group  with  confirmed  ACS  (about  8  percent 
of  apparent  cardiac  patients),  will  prove  to  have  AMI.  About  40  percent  of 
patients  found  to  have  AMI  (about  3  percent  of  the  overall  group)  will  have  early 
stage  AMI  and  can  benefit  from  reperfusion  treatment.  Thus,  the  problem  for  the 
ED  physician  is  to  promptly  and  accurately  identify,  triage,  and  treat  the  relatively 
small  proportion  of  patients  who  require  immediate  emergency  care,  while 
efficiently  dealing  with  the  great  majority  who  do  not  truly  have  ACS.  This  has 
been  a  focus  of  our  research  over  the  past  2  decades,  which  includes  the 
development  of  the  time-insensitive  predictive  instrument  (TIPI)  approach  to 
improving  ED  triage  and  treatment  decisionmaking.1  5 

Errors  are  made  in  the  care  of  such  patients;  thus,  there  are  important 
opportunities  for  improvement  in  these  ED  triage  and  treatment  decisions.  For  ED 
triage,  about  12,000  patients  with  AMI  and  14,000  with  UAP  are  mistakenly  sent 
home  each  year  in  the  United  States,  which  nearly  doubles  the  expected  mortality 
rates.6  In  light  of  this,  it  is  not  unexpected  that  cases  of  ACS  missed  by  ED 
personnel  represent  from  year  to  year  one  of  the  largest  cost  categories  of  adult 
malpractice  claims  in  the  United  States.  Regarding  ED  treatment  of  patients  with 
AMI,  the  lifesaving  impact  of  coronary  reperfusion  (especially  by  use  of 
thrombolytic  therapy)  is  directly  related  to  the  earliness  of  use.  An  hour’s  delay, 
which  is  common,  can  halve  the  mortality  benefit7, 8 — yet  many  AMI  patients  are 
not  treated  promptly,  and  about  90,000  patients  per  year  are  not  treated  at  all. 
These  errors  in  triage  and  treatment  for  ACI  are  clinically  critical  to  the  patient 
and  occur  on  a  scale  that  makes  such  errors  a  public  health  issue,  and  thus  present 
important  opportunities  to  reduce  medical  errors. 

The  Acute  Cardiac  Ischemia  Time-Insensitive  Predictive  Instrument 
Infonnation  System  (ACI-TIPI-IS)  Demonstration  Project  used  multiple 
information  technology  (IT)  applications  for  patient  safety  by  combining  real¬ 
time  decision  support,  alerting,  and  retrospective  feedback  for  performance 
improvement  for  a  single  group  of  patients:  patients  presenting  to  the  ED  for 
ACS.  A  Web-based  relational  database  system,  the  TIPI-IS,  electronically 
compiles  information  from  existing  operational  systems  about  all  ED  patients  for 
whom  an  electrocardiogram  (ECG)  was  done  and  supports  the  patient  safety 
aspects  of  the  project.  Real-time  decision  support  was  provided  through  the  use  of 
the  ACI-TIPI  electrocardiograph,  which  calculates  the  probability  (expressed  as  a 
value  from  0-100  percent)  that  the  patient  is  truly  having  ACS;  this  probability  is 
printed  on  the  top  of  the  ECG  (Figure  1). 
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Figure  1 .  Computerized  electrocardiograph  version  of  the  Acute  Cardiac  Ischemia  Time- 
Insensitive  Predictive  Instrument  (ACI-TIPI) 
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Upon  detection  of  abnormal  cardiac  biomarker  results,  the  TIPI-IS  generated 
alerts  concerning  patients  who  had  been  sent  home  from  the  ED,  supporting 
followup  review  and  action,  if  warranted.  Physician-specific  feedback  reports 
about  the  management  and  outcomes  of  patients  with  potential  ACS  were 
compiled  by  the  ACI-TIPI-IS  and  were  available  online.  These  reports  were 
provided  via  e-mail  to  staff  with  system  access  and  also  in  paper  fonnat,  to  allow 
self-evaluation  of  clinical  practice  and  to  support  oversight  of  ED  operations.  The 
goal  of  the  project  was  to  demonstrate  the  ability  of  a  completely  electronic  safety 
system  to  provide  necessary  decision  support  and  feedback  to  clinicians  by  way  of 
real-time,  concurrent,  and  retrospective  IT  systems.  The  project  made  use  of  the 
data  already  collected  in  the  course  of  patient  care  through  an  electronic  data 
collection  and  reporting  system.  Through  the  successful  implementation  and 
demonstration  of  the  system,  we  identified  barriers  and  learned  lessons  that 
inform  continued  development  of  this  technology  and  approach  to  prevent  missed 
myocardial  infarctions  in  the  ED,  and  that  serve  to  inform  further  research  in  this 
area.  This  article  describes  the  development  and  implementation  of  IT  that 
integrates  our  existing  ACI-TIPI  decision-support  technology  into  a  patient 
safety/quality  improvement  reporting  TIPI-IS  database  and  feedback  system. 


Project  overview 

As  with  our  previous  work  using  real-time  decision  support,1'5  at  the  core  of 
our  integrated  TIPI-IS  approach  is  the  use  of  a  predictive  instrument,  an  algorithm 
contained  in  the  electrocardiograph’s  software.  This  algorithm  (ACI-TIPI)  was 
used  to  compute  a  probability  (expressed  as  a  percent)  that  patients  presenting  to 
the  ED  with  complaints  of  chest  pain  or  related  symptoms  truly  had  ACS  (or, 
specifically,  of  ACI).  The  predictive  instrument’s  result — a  value  ranging  from 
0-100  percent — was  calculated  on  the  basis  of  the  ECG  waveform  measurements, 
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and  the  patient’s  age,  gender,  and  presence  of  chest  pain,  as  entered.  The  resulting 
value  is  automatically  printed  on  the  ECG’s  text  header  as  a  decision-support  aid 
for  the  physician.  In  addition  to  being  printed  on  the  ECG  header,  the  ACI-TIPI 
probability  was  stored  in  the  electrocardiograph’s  computer  and  then  transmitted 
to  the  TIPI-IS.  When  the  ECG  files  were  transmitted  to  the  TIPI-IS,  the  patient’s 
ACI-TIPI  probability  and  basic  ECG  data  became  part  of  the  ED’s  patient  safety 
database.  Added  to  this  were  data  from  other  hospital  infonnation  systems, 
including  the  patient’s  demographic  data,  cardiac  biomarker  results,  and  ICD-9- 
coded  diagnoses  and  procedures.  All  of  these  data  were  integrated  in  the  database, 
to  create  the  TIPI-IS  patient  records. 

Figure  2.  How  the  TIPI-IS  fits  into  the  hospital’s  network  of  information  systems 


The  collected  data  elements  included  those  that  allowed  us  to  define  specific 
alerts  (using  criteria  based  on  significant  clinical  variables),  along  with 
demographic  and  clinical  data  for  retrospective  reports  and  research  analyses. 
These  reports  supported  initiatives  involving  ED  staff,  leadership,  quality 
improvement  (QI),  and  resource  use. 

The  TIPI-IS  integrated  the  data  it  received  through  the  multiple  interfaces  to 
create  patient  records.  The  flexible  design  of  the  TIPI-IS  accommodates  changes 
in  transmitted  data,  such  as  the  future  conversion  to  ICD-10  coding,  without 
requiring  changes  to  the  TIPI-IS  software.  A  Web-based  interface  allowed  access 
to  individual  and  group  data  and  reports.  The  TIPI-IS  homepage  is  presented  in 
Figure  3. 
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Figure  3.  TIPI-IS  homepage 
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The  database  was  a  rich  source  of  information  that  could  be  used  to  identify 
actual  or  potential  patient  safety  issues,  and  to  support  the  quality  improvement 
cycle.  Physicians,  managers,  and  quality  improvement  staff  accessed  the  TIPI-IS 
database  through  a  Web-based  interface  that  allowed  them  to  view  and  print 
reports,  view  triggered  alerts,  enter  followup  information,  search  for  individual 
patients  and  groups  of  patients,  and  view  patient-level  details,  including  ECGs. 


Patient  safety  alerts 

Development  of  alert  criteria 

In  conjunction  with  ED  physician  leadership,  we  developed  patient  safety 
alert  criteria  to  identify  cases  thought  to  require  case  review  and  followup.  The 
alerts  included  patients  sent  home  from  the  ED  with  abnormal  cardiac  biomarker 
test  results,  and  patients  sent  home  from  the  ED  with  an  ACI-TIPI  probability 
value  that  was  moderate  (11-55  percent)  or  high  (>55  percent).  An  iterative 
process  was  required  initially  to  refine  the  alert  definitions  and  their  triggering 
criteria.  The  cardiac  biomarker  alerts,  once  triggered,  were  delivered  to  staff  at 
each  hospital  ED  for  immediate  review.  The  ACI-TIPI  alerts  were  also 
transmitted  to  the  hospitals,  but  were  scheduled  for  retrospective  case  review  by 
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the  ED  physician-director.  Each  alert  contained  information  that  described  the 
type  of  alert,  the  date  and  time  transmitted,  patient  identifiers,  and  patient  contact 
information  for  followup. 

Each  triggered  alert  was  stored  and  accessible  via  the  TIPI-IS  at  each  hospital. 
The  Alert  and  Follow-up  Web  page  displayed  a  record  of  each  alert  and  a  form  to 
collect  followup  data.  Users  who  had  appropriate  access  could  log  onto  the 
system  and  view  the  alerts  by  type,  time  period,  and  by  followup  status.  The 
results  of  reviews  of  alerts  were  entered  through  this  form  and  summary  reports 
were  available. 

Alert  delivery  methods 

The  alerting  process  provided  a  mechanism  to  let  physicians  know  when  a 
patient  was  sent  home  who  met  criteria  for  patient  safety  alerts.  When  the  TIPI-IS 
received  information  indicating  that  an  ED  patient  who  met  alerting  criteria  had 
been  sent  home,  a  message  was  sent  to  the  ED  requesting  review  of  the  case. 
These  messages  were  sent  by  e-mail,  as  a  text  message  to  a  pager,  or  to  a 
specified  printer  (Figure  4).  Each  alert  functioned  separately  from  other  alerts.  If 
positive  cardiac  biomarker  test  results  were  identified  for  a  patient  who  had  been 
sent  home,  that  alert  would  be  triggered  and  sent  to  the  ED,  whether  or  not  the 
patient  had  a  high  ACI-TIPI  probability  or  even  had  an  ECG  in  the  database.  The 
intention  of  the  alert  was  to  notify  the  staff  at  the  earliest  possible  moment  that  a 
patient  might  require  immediate  followup,  to  prompt  further  evaluation  or 
treatment. 


Figure  4.  TIPI-IS  alert 
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The  method  of  alerting  used  at  each  hospital  varied:  in  some  cases,  the 
existing  process  for  notification  about  critical  lab  values  was  used;  in  others,  a 
more  structured  delivery  process  was  put  in  place.  Some  hospitals  used  multiple 
delivery  methods,  to  ensure  that  alerts  were  received  and  followup  was 
performed. 

In  the  first  hospital  to  use  the  TIPI-IS,  alerts  were  sent  to  a  dedicated  e-mail 
address  that  was  already  used  by  the  hospital’s  Clinical  Laboratory  and  Radiology 
Departments  to  communicate  important  results  to  the  ED.  E-mail  alerts  were 
printed  by  the  ED  unit  secretary  and  handed  to  the  ED  physician  on  duty.  The 
e-mails  also  were  sent  to  the  project  staff  so  they  could  understand  the  alert 
followup  process.  The  second  participating  hospital  used  a  combination  of  text 
paging  and  e-mail  alerts.  The  text  message  was  delivered  to  a  pager  at  the  ED 
secretary’s  desk  as  well  as  to  the  QI  nurse  in  the  ED.  Both  the  physician  on  duty 
and  the  QI  nurse  were  responsible  for  followup.  At  a  third  hospital  involved  in  the 
project,  the  alerts  were  sent  by  text  message  to  the  ED  director’s  pager  who  then 
contacted  the  physician  on  duty  for  followup.  The  fourth  hospital  requested  that 
alerts  be  sent  to  a  dedicated  printer  in  the  ED,  as  well  as  to  the  hospital  QI 
coordinator  and  to  the  ED  physician-manager. 

Review  of  alerts 

Each  alert  was  reviewed  and  the  results  entered  into  the  followup  database. 
Review  of  the  alerts  resulted  in  a  greater  awareness  of  the  variability  in  managing 
patients  with  chest  pain,  issues  of  communication  between  resident  and  attending 
staff,  insufficient  documentation  of  laboratory  results  and  patient  management, 
and  the  need  for  consistent  followup  of  patients  with  positive  biomarker  results 
who  left  the  ED  against  medical  advice.  A  small  number  of  patients  returned  to 
the  ED  for  followup  and  were  admitted  for  further  care.  The  issues  identified  by 
the  ED  directors  were  addressed  with  staff  during  monthly  Morbidity  and 
Mortality  meetings  and  in  individual  feedback  discussions. 

Retrospective  reports 

The  TIPI-IS  was  used  to  provide  monthly  physician-  and  hospital-level 
feedback  reports  on  the  management  of  ED  patients  from  the  point  of  the  patient 
having  an  ECG  with  an  ACI-TIPI  probability.  Physicians  were  provided  a 
summary  of  graphical  reports  about  patients  they  treated  in  the  ED  and  their 
subsequent  disposition:  i.e.,  to  the  intensive  care  or  coronary  care  unit 
(ICU/CCU),  a  monitored  bed,  an  unmonitored  bed,  or  home.  They  could  view  the 
percentages  of  patients  they  hospitalized  or  discharged  to  home,  compared  to  the 
group  average,  and  could  view  details  about  each  patient.  Reports  also  focused  on 
patients  who  were  sent  home  with  high  ACI-TIPI  probabilities  (>55  percent)  and 
patients  admitted  to  the  CCU/ICU  with  low  ACI-TIPI  probabilities  (<10  percent). 
Demographic  data  about  patients  were  provided  along  with  ECG  results,  cardiac 
biomarker  test  results,  and  coded  ICD-9  diagnoses  and  procedures  (Figure  5). 
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Figure  5.  Average  ACI-TIPI  probability  by  ED  disposition 


As  part  of  their  training,  emergency  department  physicians  are  expected  to 
track  the  outcomes  of  patients  they  hospitalize  as  well  as  those  whom  they 
discharge  from  the  ED  and  who  subsequently  return.  Using  the  TIPI-IS  for 
hospitalized  patients,  physicians  could  view  the  sequence  of  biomarker  results  to 
determine  if  a  patient  met  criteria  for  AMI.  They  could  also  view  the  coded  ICD-9 
diagnoses  and  the  cardiac  procedures  the  patient  had  after  admission.  Through  the 
Web  site,  the  physician  could  view  the  patient’s  ECGs  throughout  their  stay, 
including  the  ACI-TIPI  probability  in  the  ED,  and  the  confirmed  ECG 
interpretation.  The  length  of  stay  is  available,  as  well  as  the  patient’s  final 
disposition.  Information  was  provided  to  ED  physician-managers  on  the  volume 
of  patients  admitted  and  the  proportion  who  were  ruled  in  for  AMI  among  those 
who  had  ECGs  and  cardiac  biomarker  tests  performed. 

Different  levels  of  feedback  reports  were  made  available  to  staff  with  different 
functions.  All  physicians  could  view  their  own  data  and  compare  results  to  the 
group  and  to  other,  unidentified  physicians.  Physician  directors  and  QI  staff  could 
view  identified  data  for  all  staff.  Followup  data  entry  was  limited  to  QI  staff  and 
ED  leaders.  These  TIPI-IS  Retrospective  Reports  included: 

•  Patients  sent  home  with  positive  biomarkers  or  high  risk  level 
ACI-TIPI  probabilities  (scores  >55  percent). 
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•  Patients  who  returned  to  the  ED  within  72  hours  and  required 
hospitalization. 

•  High-,  moderate-,  and  low-risk  patients  by  ED  triage  disposition. 

•  Patients  admitted  to  the  ICC/CCU  with  low  ACI-TIPI  probabilities. 

•  Average  ACI-TIPI  probabilities  by  ED  triage  disposition. 

Distribution  of  reports 

Reports  were  made  available  to  physicians  through  the  Web-based  TIPI-IS 
system,  as  well  as  being  sent  directly  to  their  offices.  When  a  physician  logged 
onto  the  TIPI-IS  to  view  their  reports,  a  drill-down  feature  allowed  them  to  view 
all  available  data  on  each  of  their  patients.  The  system  server  was  placed  on  the 
hospital  Intranet  and  a  TIPI-IS  link  was  located  next  to  the  ED’s  own  home  page 
link.  The  TIPI-IS  link  also  was  e-mailed  to  participating  physicians  each  month. 
Paper  versions  of  the  reports  were  distributed  monthly  through  mailings  to 
physicians’  offices,  and  they  could  access  the  same  reports  through  their 
desktop  PC. 


Site  selection 

Hospitals  that  had  (1)  ED  electrocardiographs  with  ACI-TIPI  capability, 

(2)  institutional  readiness  to  install  the  system  and  related  computer  system 
interfaces,  and  (3)  the  leadership’s  willingness  to  support  the  project  were 
identified  as  potential  sites  for  this  project.  Self-insured  hospitals  that  were  clients 
of  the  malpractice  liability  broker  Marsh,  Inc.,  a  strong  supporter  of  the  project, 
were  also  approached,  because  these  hospitals — while  motivated  to  maintain 
quality — directly  bear  the  financial  risk  of  missed  AMI.  Electrocardiographs  with 
ACI-TIPI  decision-support  software  are  available  through  the  two  largest 
electrocardiograph  vendors.  A  third  manufacturer,  which  made  the 
electrocardiograph  model  used  by  one  of  the  hospitals,  developed  the  software  in 
response  to  their  customer’s  request  to  participate  in  the  study  and  will  continue 
to  make  the  software  available  to  the  hospital. 

In  discussing  the  project  with  potential  participants,  we  found  it  challenging  to 
reach  consensus  among  parties  within  a  hospital.  In  some  cases,  hospital  leaders 
were  interested  in  participating  and  implementing  patient  safety  systems,  but  ED 
leadership  did  not  believe  they  were  at  risk  for  missing  patients  with  ACS.  In 
other  instances,  ED  leadership  was  willing  to  participate,  but  they  were  unable  to 
motivate  their  colleagues  in  the  cardiology  department.  Cardiology  departments  in 
some  hospitals  were  interested  in  installing  the  decision  support  software,  but  the 
same  institutions’  EDs  didn’t  believe  that  they  needed  to  improve  their 
management  of  patients  with  chest  pain  or  ACS.  In  one  case,  in  a  hospital  that  did 
not  participate  in  the  study,  all  clinical  and  administrative  parties  were  motivated, 
but  the  IT  staff  did  not  believe  they  had  adequate  resources.  Of  the  many  sites 
considered  for  participation,  the  most  common  barrier  to  participation  was 
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reaching  consensus  among  clinical  groups,  and  least  frequent  barrier  was 
technical  issues  related  to  system  interfaces  or  IT  resources. 


Development,  implementation, 
and  testing  of  the  TIPI-IS 

Hardware,  software,  and  architecture 

Each  participating  hospital  received  a  TIPI-IS  server  to  handle  all  the  local 
data  aggregation,  reporting  tasks,  and  provide  Web-based  access  to  the  TIPI-IS 
ED  patient  database.  Each  of  these  local  TIPI-IS  systems  were  further  linked  to 
the  central  TIPI-IS  datacenter  server  over  a  secure  Internet  connection.  The 
central  database  aggregated  de-identified  data  across  hospitals  for  multihospital 
benchmarking  reports. 

To  enable  system  flexibility  across  diverse  hospital  ED  project  sites,  from 
small  rural  clinics  (with  as  few  as  two  beds  in  their  EDs)  to  large  regional 
teaching  hospitals,  the  TIPI-IS  used  standard  technologies  (object-oriented 
programming,  Java™,  XML,  SQL  database)  and  communication  standards  (e.g., 
HL7,  HTTP,  TCPIP),  as  well  as  standard  approaches  from  financial  and 
e-commerce.  The  TIPI-IS  architecture  and  technologies  functioned  without  regard 
to  the  specific  operating  system  and  hardware  in  each  hospital’s  IT  environment. 
To  keep  costs  low,  each  hospital  implemented  the  TIPI-IS  using  commonly 
available  hardware  (Windows®  2000  Server  on  PC  servers)  and  proven  open 
source  software  for  its  Web  server  (Apache. org’s  Tomcat  Webserver)  and 
relational  database  (MySQL  3.23).  The  TIPI-IS  application  clients  were  standard 
Web-browsers  (Microsoft®  Internet  Explorer  5.x  and  Netscape®3),  available  on  all 
PCs  within  each  hospital. 

The  local  TIPI-IS  server  aggregated  hospital  admission/discharge/transfer 
(ADT)  data,  cardiac  laboratory  results,  and  ECG  data  from  hospital  information 
operational  systems  (using  HL7  interface  engines  and  ECG  management  systems) 
through  standard  data  interfaces  (HL7  and  FTP).  The  TIPI-IS  linked  the  patient 
data  elements  into  a  local  SQL  database  for  concurrent  alerts,  retrospective 
reporting,  and  single-point,  Web-based  access  to  the  ED  patient  profile.  Although 
nurses  and  physicians  were  able  to  access  all  these  data  elements  from  existing 
clinical  information  systems,  the  TIPI-IS  provided  a  single  point  of  access  to  view 
ED  patient  demographics,  ICD-9  diagnoses  and  procedure  codes,  ED  encounter 
and  hospital  data  (including  length  of  stay  and  repeat  visits),  and  ECG  waveform 
and  analysis  results  (including  ACI-TIPI  scores). 

System  installation 

Installation  of  the  TIPI-IS  at  each  site  included  seven  steps: 

1 .  Site  assessment  and  development  of  a  detailed  installation  and  testing 
plan. 
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2.  Customizing  of  the  TIPI-IS  to  address  hospital-specific  operational  and 
technical  needs. 

3.  Installation  of  the  TIPI-IS  server  on  the  hospital’s  Intranet. 

4.  Preparation  of  the  hospital’s  ECG  equipment. 

5.  Linking  and  integrating  each  hospital’s  key  databases  to  the  TIPI-IS 
server  via  electronic  interfaces. 

6.  Testing  interface  data. 

7.  User  training. 

A  detailed  site  assessment  was  conducted,  and  an  installation  plan  was 
developed  with  each  participating  hospital  to  address  their  technical  needs  and 
available  resources.  Each  site  identified  key  data  dictionaries  and  completed  a 
questionnaire  about  the  process  of  care  delivery  in  the  ED.  Modifications  were 
made  to  the  TIPI-IS  to  address  unique  IT  needs,  including  special  processing  of 
account  numbers  and  filtering  of  interface  data.  Hospital-specific  alerting  criteria 
and  feedback  report  parameters  were  also  programmed. 

Hospitals’  IT  departments  received  detailed  specifications  for  each  variable  to 
be  transmitted  to  the  TIPI-IS,  using  standard  HL7  transaction  protocols  and  data 
formats.  Many  hospitals  already  use  these  HL7  interface  engines  to  transmit  data 
across  internal  systems  and,  whenever  possible,  the  hospitals  used  existing 
interfaces  to  avoid  repeated  effort.  Tests  were  conducted  of  interfaces  from  feeder 
systems  to  the  TIPI-IS  at  each  site  to  ensure  that  the  system  captured  the  data 
completely  and  mapped  key  data  elements  accurately  for  reporting. 

Training  for  nursing  technicians  and  nurses  consisted  of  a  brief  overview  of 
the  project  and  a  hands-on  demonstration  of  the  use  of  the  ACI-TIPI  software  in 
the  ED  electrocardiograph.  Training  materials  were  provided,  and  simple 
instructions  were  placed  on  each  electrocardiograph  for  staff  reference.  Physicians 
received  an  overview  of  the  project  in  a  group  meeting  and  samples  of  the  ACI- 
TIPI  ECGs  and  details  on  the  development  and  testing  of  the  ACI-TIPI  in  the 
form  of  research  publications.  ED  managers  and  QI  staff  were  trained  in  the  use 
of  the  TIPI-IS  and  data  entry  of  followup  information. 

Data  collection 

Data  were  collected  at  each  site  for  a  6-month  baseline  period,  followed  by  a 
12-18  month  intervention  period.  The  advantage  of  the  electronically  collected 
database  was  that  staff  did  not  need  to  identify  cases  and  perform  manual  data 
collection  and  data  entry.  Once  the  system  was  implemented,  all  cases  meeting 
data  collection  criteria  were  automatically  identified  and  all  appropriate  data 
collected.  This  process  allowed  us  to  compile  a  database  of  more  than  415,000 
patient  encounters  across  the  five  hospitals,  including  130,000  ECGs  with  ACI- 
TIPI  probability  scores.  The  inclusion  of  ICD-9  diagnostic  codes  and  cardiac 
biomarker  results  allowed  us  to  provide  analyses  of  patient  outcomes. 
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The  creation  of  the  TIPI-IS  proved  to  be  an  effective  method  for  quickly 
compiling  data  into  one  system  for  use  in  a  patient  safety  system.  The  compiled 
information  was  not  available  to  these  hospitals  in  any  other  single  system, 
including  the  institution’s  electronic  medical  record.  Criteria  for  initiating  alerts 
were  applied  continually  to  the  transmitted  data,  and  alerts  were  triggered  24 
hours  a  day,  without  requiring  staff  intervention.  Feedback  reports  were  compiled 
at  the  conclusion  of  each  month  and  posted  on  the  system. 

Collecting  data  through  automated  interfaces  had  certain  disadvantages:  the 
need  for  detailed  tracking  of  data  quality  for  large  data  sets;  and  the  likelihood  of 
unanticipated  disruptions  in  data  transmission  to  the  system,  due  to  system 
downtime,  upgrades,  and  operational  changes  to  interfaces  or  feeder  systems  that 
affect  the  interface.  Part  of  the  preinstallation  assessment  process  involved 
working  with  the  local  IT  department  representatives  to  identify  requirements  for 
disaster  recovery  and  downtime  procedures,  and  adherence  to  hospital  data  center 
policies  and  procedures.  The  local  hospital  IT  group  provided  the  same 
uninterruptable  power  services,  backup  and  recovery  services,  antivirus 
protection,  and  constant  (24-hour/7-day)  server  monitoring  services  to  the  TIPI-IS 
that  it  would  to  any  other  clinical  and  administrative  data  server  operating  within 
the  hospital  data  center  facilities.  While  HL7  interfaces  are  fairly  standardized, 
they  are  interpreted  differently  by  different  organizations,  and  processing  of  the 
same  type  of  data  may  differ  among  hospitals.  If  a  third-party  vendor  provides  IT 
services,  the  internal  IT  department  may  have  limited  knowledge  about  the  system 
and  may  have  to  pay  the  vendor  for  even  minor  interface  changes. 


Successes  and  lessons  learned 

Real-time  decision  support 

The  ACI-TIPI  software  was  available  at  or  supplied  to  each  of  the 
participating  hospitals,  the  staff  was  trained  in  its  use,  and  the  completeness  of  the 
data  was  monitored.  Approximately  50-85  percent  of  the  ED  ECGs  had  complete 
real-time  data  entry,  such  that  the  ACI-TIPI  probability  was  printed  on  the  ECG 
for  real-time  decision  support.  It  is  not  clear  what  the  optimal  proportion  with 
complete  data  should  be.  Not  all  ECGs  in  an  ED  are  performed  for  suspicion  of 
ACS,  in  which  case  the  ACI-TIPI  probability  would  not  be  indicated. 

Nonetheless,  initially,  available  data  on  patients’  presenting  symptoms  suggested 
that  many  patients  for  whom  the  ACI-TIPI  would  be  appropriate  did  not  have  the 
required  data  entered  (age,  sex,  and  chest  pain  status).  Actions  were  taken  at  each 
hospital  to  improve  the  completeness  of  the  ACI-TIPI  data  entry,  including 
feedback  to  nurses  and  technicians  about  missing  and  incorrect  data. 

The  project  illustrated  an  important  consideration  in  developing  future 
decision-support  models  for  real-time  use  by  clinicians.  While  age  and  sex  are 
fairly  straightforward  variables,  the  patient’s  chest  pain  status  is  more  likely  to  be 
entered  incorrectly,  due  to  human  error  on  the  part  of  the  technician  or  patient. 
The  chest  pain  status  identities  the  absence  or  presence  of  chest  pain  as  a  primary 
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or  secondary  reason  for  coming  to  the  ED.  By  “chest  pain,”  we  mean  a  family  of 
related  symptoms,  including  left  arm  pain,  jaw  pain,  shortness  of  breath,  upper 
abdominal  pain,  and  dizziness  or  syncope.  Potential  errors  include 
misinterpretation  by  the  technician  of  the  meaning  of  the  chest  pain  variable, 
misinterpretation  of  the  patient’s  response,  patient  misinterpretation  of  the 
question,  or  patient  provision  of  incorrect  or  incomplete  information.  Since  the 
chest  pain  status  variable  plays  an  important  part  in  calculating  the  ACI-TIPI 
probability,  the  inevitable  contribution  of  error  by  the  patient  or  technician 
affected  the  score’s  accuracy  and  relevance  for  use  in  real-time  and  retrospective 
feedback  reports. 

Future  development  of  decision-support  models  should  take  the  potential 
contribution  of  patient  or  technician  error  into  account  by  minimizing  or 
eliminating  the  opportunities  or  effect  of  human  error  in  the  algorithm. 

Concurrent  alerts 

The  system  for  generating  concurrent  alerts  was  successfully  implemented  at 
each  hospital,  using  criteria  supported  by  that  hospital’s  ED  physicians  and 
delivery  methods  appropriate  to  that  site.  The  criteria  and  thresholds  for  alerts 
were  specific  to  the  needs  of  the  organization  to  engender  buy-in  by  users.  The 
levels  of  false-positive  diagnosis  on  triage  (unnecessary  hospitalization)  or  the 
potential  for  false  negatives  (failure  to  hospitalize  a  patient  with  ACS)  were 
concerns  when  physicians  decided  on  the  alerting  criteria.  Once  the  alerting 
process  was  in  place,  adjustments  in  the  criteria  were  made  to  account  for  higher- 
than-anticipated  rates  of  false-positive  alerts.  False-negative  alerts  were  difficult 
to  identify,  since  there  was  no  systematic  method  for  following-up  on  all  patients 
sent  home.  We  devised  a  substitute  method  for  identifying  potential  false 
negatives  by  screening  the  charts  of  patients  sent  home  with  moderate  and  high 
ACI-TIPI  probability  scores. 

While  the  methods  of  delivering  alerts  varied  in  their  success,  the  key  to 
adequate  followup  was  the  presence  of  an  invested  third  party,  or  champion,  with 
the  responsibility  for  alert  followup.  The  e-mail  alert  process  was  effective  in 
delivering  the  message,  but  successful  followup  relied  on  the  unit  secretary  to 
deliver  the  e-mail  to  the  ED  physician  on  duty.  A  text  pager  at  a  secretary’s  desk 
was  not  an  effective  method  to  deliver  alerts  in  a  busy  and  complex  setting, 
because  the  alerts  were  infrequent  and  the  pager  could  be  misplaced.  Alerts 
communicated  through  a  text  pager  in  the  hands  of  the  responsible  physician  were 
effectively  delivered,  but  false-positive  messages  due  to  incorrect  identification 
by  staff  of  the  patient  as  being  sent  home  from  the  ED  were  bothersome. 

A  three-level  alert  notification  process,  including  e-mail  messages  to  QI  staff 
and  the  ED  manager,  as  well  as  a  printed  alert  in  the  ED,  proved  to  be  the  most 
effective.  This  process  ensured  that  those  responsible  for  followup,  as  well  as  the 
ED  manager,  were  aware  of  the  alert  and  could  ensure  the  followup  took  place. 
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Feedback  reports 

The  feedback  reports  were  compiled  and  made  available  through  the  Web- 
based  system  shortly  after  the  conclusion  of  the  given  reporting  period,  and  an 
archive  of  all  reports  were  available  online  to  all  ED  physicians  at  each  hospital. 
In  an  anonymous  survey  of  physicians  at  one  site,  66  percent  of  physicians 
reported  reviewing  the  reports  and  identifying  their  own  performance  among  the 
group  of  physicians  displayed  on  the  report. 

The  disadvantage  of  the  Web-based  system  was  the  perception  that  it  was 
difficult  to  access  the  system  to  view  the  reports.  Alternative  forms  of  feedback 
report  delivery  were  tried.  Paper  versions  of  the  reports  were  distributed  monthly, 
by  mail,  to  each  physician’s  office.  An  e-mail  message  with  a  direct  link  to  the 
Web-based  reports  was  devised  that  allowed  easier  distribution  and  access  to 
physician  reports,  and  avoided  the  need  for  the  physician  to  log  onto  the  system 
while  still  ensuring  the  security  of  the  system.  The  e-mail  message  focused  the 
physicians’  attention  on  a  few  key  reports  rather  than  the  large  number  of  reports 
originally  available.  These  reports  were  geared  to  helping  the  physicians  followup 
on  the  outcomes  of  patients  who  had  been  hospitalized,  as  well  as  those  who  were 
sent  home  and  who  later  returned  to  the  ED. 


Conclusion 

This  ACI-TIPI-  and  TIPI-IS-based  project  aimed  at  reducing  errors  in 
emergency  cardiac  care  illustrates  the  use  of  usual  clinical  IT  (conventional 
computerized  electrocardiographs  with  ACI-TIPI  software)  and  existent  hospital 
IT,  along  with  conventional  PC-based  and  interface  IT.  The  project  successfully 
demonstrated  that  a  patient  safety  system  using  a  completely  electronic  data 
collection  and  feedback  reporting  system  and  offering  real-time  decision  support, 
concurrent  patient  safety  alerts,  and  retrospective  physician  level  feedback  reports 
could  be  implemented  in  a  variety  of  hospital  settings.  Given  the  current  health 
care  system  in  the  United  States,  it  is  certain  that  successful  dissemination  of  the 
technology  will  require  some  form  of  commercialization.  The  optimal  structure 
and  process  for  doing  this  is  yet  to  be  detennined.  While  there  is  clearly  great 
opportunity  for  generic  cross-cutting  IT  approaches  to  patient  safety,  in  the  short 
tenn,  or  in  addition,  we  believe  our  project  illustrates  the  potential  of  condition¬ 
er  care-specific  IT-based  patient  safety  systems.  Such  targeted,  limited,  and  less 
costly  approaches  may  well  be  an  important  component  of  patient  safety  and  QI 
activities  and  deserve  further  investigation  in  the  wide  variety  of  health  care 
settings  so  in  need  of  such  efforts. 
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